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DETAILED ACTION 
Response to Amendment 

1. This office action is in response to communications filed 2/8/201 1. Claims 1-3, 1 1-18 and 
20 are amended. Claims 4-6 and 19 are cancelled. Claim 22 is new. Claims 1-3, 7-18 and 20-22 
are pending in this action. 

Response to Arguments 

2. Applicant's arguments filed 2/8/2011 have been fully considered but they are not 
persuasive. 

3. In response to Applicant's arguments on page 10, lines 3-27, regarding claim 1 that, 
"Bulman's personalized video tape does not teach "receiving a first set of computer generated 
views of the preregistered picture in various orientations at a video receiver before the plurality 
of video images is received" because a photograph resized into several sizes is not a first set of 
computer generated views of the preregistered picture in various orientations. Different sizes are 
not different orientations. The Examiner acknowledges this distinction and refers to a second 
embodiment (Example 6) of Bulman. See Office Action, p. 3. 

In that embodiment, Bulman describes recording a human subject head with a video 
camera. The subject stands on a turntable which is rotated. The images recorded by the video 
camera differ in orientation and positioning. See col. 13, lines 28-42. However, these images are 
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not a first set of computer generated views of the preregistered picture in various orientations at 
a video receiver. A video of a human subject cannot be reasonably considered a set of computer 
generated views. 

Bulman also states in a third embodiment (Example 7) that a library of images may 
include "images varying in perspective, size, action (e.g., mouth open, mouth closed, eyes open, 
eyes closed, smiling, frowning, etc.), lighting, or other characteristics." See col. 12, lines 18-25. 
However, the library of images cannot be considered a first set of computer generated views of 
the preregistered picture in various orientations because they are not computer generated. 
Bulman specifically states that the "image library may also be supplemented with synthesized 
images," which distinguishes the other images mentioned previously as not synthesized, and 
further states that even the synthesized are "interpolated or extrapolated from actual images." In 
either case, this embodiment of Bulman cannot be reasonably construed to teach "receiving a 
first set of computer generated views of the preregistered picture in various orientations at a 
video receiver before the plurality of video images is received", and similar subsequent 
arguments regarding claim 12, the Examiner respectfully disagrees. Bulman discloses that the 
image of the head and/or head of his system may be obtained from a "real" head or body, 
respectively, either photographically or by electronic image scanning, or from an artistic or 
computer generated rendering thereof (see Bulman, col 1, lines 27-30). Bulman further discloses 
video frames of the head being obtained and stored, and thereafter digitized to provide digital 
representations of each frame (see Bulman, col 8, lines 9-20), or, alternatively, using a scanner to 
electronically scan a photograph and produce digital representations defining the image, wherein 
perspective views of the subject may be artificially generated from one or more views of the 
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subject (see Bulman, col 8, lines 25-28). The representations of the subject resulting from the 
methods as disclosed above are all generated using a computer. Therefore, the limitation as 
claimed is met. 

In response to Applicant's arguments on page 10, line 28 - page 11, line 10 that, "In 
addition, nothing in Bulman suggests that the views are received at a video receiver before the 
plurality of video images is received. In the embodiment of the personalized videotape (Example 
5), the PC workstation retrieves the views and combines it with a video story. See col. 1 1, lines 
30-36. However, Bulman does not state that the views are scaled before the video is received. 
Finally, the combined image is stored with a VCR, which suggests that is not superimposed on 
video images as the video images are received. 

Moreover, in the second embodiment (Example 6), Bulman expressly requires that this 
embodiment allows "production to occur in real time and be synthesized, sequenced and directly 
recorded on a video tape" and "the facial image to be scaled, rotated, and placed in real time on 
the background video image." See col. 12, lines 14-38. If the facial images are scaled and 
rotated in real time, Bulman does not describe "receiving a first set of computer generated views 
of the preregistered picture in various orientations at a video receiver before the plurality of 
video images is received", and similar subsequent arguments regarding claim 12, the Examiner 
respectfully disagrees. Bulman discloses that his digital representations of the image are stored in 
memory 102 and may include representations of varying perspective, size, action, etc. (see 
Bulman, col 12, lines 19-24), and that the proper image representation, i.e., foreground image, is 
retrieved from the memory and scaled, rotated and placed on the background image (see Bulman, 
col 12, lines 36-38) in real time (see Bulman, col 12, lines 14-18 and line 37) based on SMPTE 
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time codes and other associated information regarding the optimum location and orientation of 
the image (see Bulman, 29-40). The scaling and rotating done may be in addition to any 
processing done before the digital representation were stored, since a desired positioning and 
orientation for a cohesive match with the background image may be obtained by selecting the 
closest image actually obtained, and then further transformed by linear scaling, interpolation, or 
other processing (see Bulman, col 13, lines 29-38). Since the digital representations are already 
stored, they may reasonably be received before the background images. 

In response to Applicant's arguments on page 12, line 16 - page 13, line 2 regarding 
claim 20 that, "The Office Action acknowledges that Lemmons and Bulman do not "disclose 
transmitting the enhancement information in advance of transmission of the stream of video 
images." See Office Action, p. 18. However, the Office Action asserts that Lemmons '981 teaches 
this feature. Assignee respectfully disagrees. 

Claim 20 does not recite "enhancement information" as characterized by the Office 
Action. Claim 20 recites "a first set of computer generated views of preregistered picture in 
various orientations." Lemmons '981 describes a system where a television program is 
transmitted on a first channel and enhancements to the program are transmitted on a second 
channel. See abstract. The enhancements may include management messages or program guides. 
See f 18. The receiver may store combined data in a hard drive. See f 29. 

Management messages or program guides cannot be reasonably considered a set of 
computer generated views associated with an orientation index. Therefore, Lemmons '981 does 
not suggest, "in advance of transmission of the stream of video images, transmitting each 
computer generated view of the first set of computer generated views in association with an 
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orientation index that identifies a physical orientation of the computer generated view of the 
preregistered picture," as recited by claim 20. 

Therefore, none of Lemmons, Bulman, Lemmons '981, or combinations thereof teach or 
suggest all of the features of claim 20. Accordingly, Assignee respectfully requests that the 
rejections of claim 20 and dependent claim 21 be withdrawn", and similar arguments regarding 
the newly added limitations to claims 1 and 12, the Examiner respectfully disagrees. The system 
of Lemmons in view of Bulman teaches providing enhancements to video content (see 
Lemmons, [0003], and Bulman, col 6, lines 6-10) in a more normalized (see Bulman, col 1, lines 
60-63), flexible and inexpensive manner (see Lemmons, [0006] and [0007]) in that the 
enhancement information does not have to be embedded within the video stream (see Lemmons, 
[0010]). Lemmons '981 teaches enhancement data being transmitted and received before the 
video stream to be enhanced is transmitted and received (see Lemmons '981, [0029]-[0030], 
lines 1-4). Lemmons '981 teaches enhancement data including stock updates and news stories, as 
well as other information, as examples only. It is well known in the art that enhancement data 
may be of any type or form, as shown in Lemmons and Bulman. Therefore, it would have been 
obvious for a person having ordinary skill in the art at the time of the invention to modify the 
system of Lemmons in view of Bulman to include the limitations as taught by Lemmons '981 for 
the advantage of providing an improved system for delivering enhancements such that the 
enhanced video is presented in a more normalized, flexible and inexpensive manner, while 
conserving bandwidth. 

In response to Applicant's arguments on page 13, lines 7-19 regarding claim 21 that, 
"Claim 21 recites transmitting a polygon representation of an obstruction with the video image 
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and the selected orientation index. The Office Action asserts Wang teaches the subject matter of 
Claim 21. Specifically, the Office Action reasons that "any object in the camera's field of view, 
including an obstruction, will be represented in the transmitted signal." Assignee respectfully 
disagrees with this reasoning. Wang does not suggest a polygon representation of an 
obstruction. The mere possibility that a filmed object could be considered an obstruction does 
not rise to the level of transmitting a polygon representation of an obstruction. For at least these 
reasons, Assignee submits that none of the cited prior art teaches or suggests transmitting a 
polygon representation of an obstruction with a video image and an orientation index. 
Accordingly, Applicants submit that, in addition to the reasons above, claim 21 is allowable over 
the cited prior art", the Examiner respectfully disagrees. Wang teaches creating a view of a 
synthetic camera based on the field of view of an actual camera (see Wang, col 7, lines 37-39). 
Any object in a camera's field of view, then, including an obstruction, will be represented in the 
transmitted signal (see Wang, col 7, lines 37-54). Furthermore, objects presented in the synthetic 
view may be three dimensional graphic representations of an object (see Wang, col 8, lines 43- 
52), which, then, must include polygons since virtual and/or 3D graphics may be built based on 
various polygons. Therefore, the limitation as claimed is met. 

In response to Applicant's arguments on page 13, lines 25-27 regarding claim 22 that, 
"None of the cited prior art teaches or suggest video images received with a corresponding 
orientation index and superimposing a computer-generated view having the same orientation 
index on each of the video images", the Examiner respectfully disagrees. Bulman teaches that his 
background images are coded with SMPTE time codes and other information regarding the 
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optimum location and orientation of the image (see Bulman, col 12, lines 29-33 and col 14, lines 
6-9). Therefore, the limitation as claimed is met. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

5. Claim 22 is rejected under 35 U.S.C. 102(e) as being anticipated by Bulman (of record). 

Regarding claim 22, Bulman discloses a method of superimposing (see Bulman, col 2, 
lines 16-18) a preregistered picture (head/foreground image, see Bulman, col 1, lines 27-30, col 
2, lines 16-18 and col 12, lines 21-25) on a predetermined area of the image of a moving object 
(see Bulman, col 5, lines 26-34) in a plurality of video images (see Bulman, col 5, lines 8-18), 
the method comprising: 

receiving and storing in a memory (see Bulman, col 12, lines 21-28) a set of computer- 
generated views of a preregistered picture (see Bulman, col 1, lines 27-30 and col 8, lines 9-28) 
each associated with a unique orientation index identifying the physical orientation of the 
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corresponding computer-generated view (see Bulman, col 11, lines 22-29, col 12, lines 21-28 
and col 13, lines 29-33); and 

superimposing on each of said video images (see Bulman, col 2, lines 16-18 and col 12, 
lines 36-38), each being received with a corresponding orientation index (Bulman teaches that 
his background images are coded with SMPTE time codes and other information regarding the 
optimum location and orientation of the image (see Bulman, col 12, lines 29-33 and col 14, lines 
6-9), the computer-generated view having the same orientation index (see Bulman, col 13, lines 
29-37), at a location and scaling indicated by size and location information transmitted with each 
video image (see Bulman, col 12, lines 29-40 and col 13, lines 33-37). 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1-3, 7-9, 11-16, 18 and 20-21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Lemmons (of record) in view of Bulman (of record), Wang (of record) and 
Lemmons '981 (of record). 
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Regarding claims 1, Lemmons discloses a method for generating a stream of video 
(scenes/frames of the video, see Lemmons, [0031], lines 1-2 and Figs. 6A and 6B) such that a 
preregistered picture (labels 610/620, see Lemmons, Figs. 6A and 6B) is superimposed on a 
predetermined area of a moving object in a plurality of video images (soda can 608/618, see 
Lemmons, Figs. 6A and 6B), the method comprising: 

determining in each of the plurality of video images a location, an orientation and a size 
of said predetermined area of said moving object (see Lemmons, [0081]-[0084], line 5), and 

using the selected information on the location and size of said predetermined area of said 
moving object to superimpose the preregistered picture on the plurality of video images to 
generate the stream of video (see Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and 
page 12, lines 1-3). 

Lemmons does not specifically disclose receiving a first set of computer generated views 
of the preregistered picture in various orientations at a video receiver before the plurality of 
video images are received, 

associating each computer generated view of the first set of computer generated views 
with an orientation index that identifies the physical orientation of the computer generated view 
of the preregistered picture, 

storing, in a machine-readable medium, the first set of computer generated views and the 
associated orientation index, 

receiving orientation and position data for the moving object captured by a sensor 
attached to the moving object, 

receiving position data for a camera that captured the plurality of video images, 
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wherein orientation is determined using the orientation and position data for the moving 
object and the position data for the camera, 

selecting, from the orientation indices associated with the stored computer generated 
views, the orientation index of the computer generated view having the same orientation as said 
predetermined area of said moving object, or 

using the selected orientation index along with information on the location and size of 
said predetermined area of said moving object to superimpose the preregistered picture on the 
plurality of video images to generate the stream of video. 

In an analogous art relating to a system for producing an electronic image, Bulman 
discloses receiving (at memory 102, see col 12, lines 21-22) a first set of computer generated 
(see Bulman, col 1, lines 27-30 and col 8, lines 9-28) views of the preregistered picture in 
various orientations at a video receiver (see Bulman, col 12, lines 21-25 and col 13, lines 29-33), 

associating each computer generated view of the first set of computer generated views 
with an orientation index that identifies the physical orientation of the computer generated view 
(see Bulman, Fig. 11 and col 12, lines 21-25 and col 13, lines 29-33), 

storing, in a machine-readable medium, the first set of computer generated views (see 
Bulman, col 12, lines 21-25 and col 13, lines 29-33) and the associated orientation index (see 
Bulman, col 13, lines 29-33), 

selecting, from the orientation indices associated with the stored computer generated 
views, the orientation index of the computer generated view having the same orientation as a 
predetermined area of an object (see Bulman, col 13, lines 33-37 and Fig. 11.), and 
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using the selected orientation index along with information on the location and size of 
said predetermined area of said moving object to superimpose the preregistered picture on the 
plurality of video images to generate the stream of video (The orientation index of Bulman's 
system includes orientation and positioning information along with the different orientated views 
of the picture, see Bulman, col 13, lines 33-37. This then, reasonably corresponds to the data file 
722 of Lemmons' system, which is transmitted with each image. Therefore, Lemmons in view of 
Bulman reasonably teaches the limitation of transmitting with each current image the selected 
orientation index along with information on the location and size of said predetermined area of 
said moving object (see Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and page 12, 
lines 1-3, and Bulman, col 13, lines 29-33). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify Lemmons' system to include the limitations as taught by Bulman for the 
advantage of providing an improved system for allowing a combined image to be displayed with 
a more natural look. 

Lemmons in view of Bulman does not specifically disclose receiving the views at the 
video receiver before the plurality of video images is received, 

receiving orientation and position data for the moving object, the orientation and position 
data captured by a sensor attached to the moving object, 

receiving position data for a camera that captured each current image, or 

wherein orientation is determined using the orientation and position data for the moving 
object and the position data for the camera. 
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In an analogous art relating to a system for providing enhanced broadcasting, Wang 
discloses receiving orientation and position data from a moving object (see Wang, col 7, lines 
26-59), the orientation and position data captured by a sensor attached to the moving object (see 
Wang, col 7, lines 55-59), 

receiving position data for a camera that captured each current image (see Wang, col 7, 
lines 8-12 and lines 60-66), and 

wherein orientation is determined using the orientation and position data for the moving 
object and the position data for the camera (see Wang, co 8, lines 12-16). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman to include the limitations as 
taught by Wang for the advantage of providing an improved system for allowing a combined 
image to be displayed with a more natural look, and further allowing the possibility to 
successfully integrate properly placed, scaled and computer generated synthetic content with 
video content. 

Lemmons in view of Bulman and Wang does not specifically disclose receiving the 
views at the video receiver before the plurality of video images is received. 

In an analogous art relating to a system for presenting an enhanced image, Lemmons 
'981 discloses transmitting enhancement information in advance of transmission of a stream of 
video images (see [0029]-[0030], lines 1-4). The system of Lemmons in view of Bulman teaches 
providing enhancements to video content (see Lemmons, [0003], and Bulman, col 6, lines 6-10) 
in a more normalized (see Bulman, col 1, lines 60-63), flexible and inexpensive manner (see 
Lemmons, [0006] and [0007]) in that the enhancement information does not have to be 
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embedded within the video stream (see Lemmons, [0010]). Lemmons '981 teaches enhancement 
data being transmitted and received before the video stream to be enhanced is transmitted and 
received (see Lemmons '981, [0029]-[0030], lines 1-4). Lemmons '981 teaches enhancement 
data including stock updates and news stories, as well as other information, as examples only. It 
is well known in the art that enhancement data may be of any type or form, as shown in 
Lemmons and Bulman. 

Therefore, it would have been obvious for a person having ordinary skill in the art at the 
time of the invention to modify the system of Lemmons in view of Bulman to include the 
limitations as taught by Lemmons '981 for the advantage of providing an improved system for 
delivering enhancements such that the enhanced video is presented in a more normalized, 
flexible and inexpensive manner, while conserving bandwidth. 

Regarding claims 2, Lemmons in view of Bulman, Wang and Lemmons '981 discloses 
providing (see Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and page 12, lines 1-3) 
at least one second set of views of a second preregistered picture (the second set of views are the 
updates to the hot spots/labels, see Lemmons, [0010], lines 10-18, [0013], lines 6-14, [0040], 
lines 6-7 and Figs. 6A and 6B), and therefore would correspond to updating the views of the 
preregistered picture of Bulman, see Bulman, col 12, lines 21-25 and col 13, lines 29-33), 
corresponding to the first set of computer generated views (the second set of views are the 
updates to the hot spots/labels, see Lemmons, [0010], lines 10-18, [0013], lines 6-14, [0040], 
lines 6-7 and Figs. 6A and 6B, and therefore would correspond to updating the views of the 
preregistered picture of Bulman, see Bulman, col 12, lines 21-25 and col 13, lines 29-33), and for 
each transmitted current image: 
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extracting the orientation index and the size and location information (see Bulman, col 
13, lines 33-37), 

selecting, from the second set of views (the second set of views are the updates to the hot 
spots/labels, see Lemmons, [0010], lines 10-18, [0013], lines 6-14, [0040], lines 6-7 and Figs. 6A 
and 6B, and therefore would correspond to updating the views of the preregistered picture of 
Bulman, see Bulman, col 12, lines 21-25 and col 13, lines 29-33), an computer generated picture 
in accordance with the orientation index (see Bulman, col 13, lines 33-37 and Fig. 11), 

computing a scaled picture on the basis of the size information (see Lemmons, [0082]- 
[0084], line 5, Figs. 6A-6B and '745, Fig. 7 and page 5, lines 1-9), and 

superimposing the scaled picture in the current image at a location corresponding to the 
location information (see Lemmons, Figs. 6A-6B and [0082]-[0083], line 5, and '745, Fig. 7 and 
page 5, lines 1-9). 

Regarding claim 3, Lemmons in view of Bulman and Wang discloses, at the beginning of 
a TV program to be transmitted, the second set of views (the second set of views are the updates 
to the hot spots/labels, see Lemmons, [0010], lines 10-18, [0013], lines 6-14 and [0040], lines 6- 
7, and therefore would correspond to updating the views of the preregistered picture of Bulman, 
see Bulman, col 12, lines 21-25 and col 13, lines 29-33) is downloaded in the video receivers (As 
the labels/label information may be downloaded, and it is very well known in the art and 
common that a download can occur at any time, Lemmons fairly suggests that at the beginning 
of a TV program to be transmitted, the second set of views is downloaded in video receivers, see 
Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and page 12, lines 1-3). 
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Regarding claim 7, Lemmons in view of Bulman, Wang and Lemmons '981 discloses the 
content of the second set of views depends upon the geographic broadcasting zone (see 
Lemmons, [0013], lines 6-14 and [0040], lines 1-13). 

Regarding claim 8, Lemmons in view of Bulman, Wang and Lemmons '981 discloses the 
location and orientation information in a current image are calculated for a reference point of the 
object (see Lemmons, [0083] -[0084], line 5). 

Regarding claim 9, Lemmons in view of Bulman, Wang and Lemmons '981 discloses, in 
a current image, the location, orientation and size of an object are provided in a differential way 
with respect to a former image (see '745, page 13, line 30 - page 14, line 11). 

Regarding claim 11, Lemmons in view of Bulman, Wang and Lemmons '981 discloses 
using shape recognition tools to detect the presence of the moving object on the basis on a stored 
geometrical representation (see Lemmons, [0064], lines 20-32). 

Regarding claim 12, Lemmons discloses a system for generating a stream of video to be 
broadcasted (scenes/frames of the video, see Lemmons, [0031], lines 1-2 and Figs. 6A and 6B) 
such as, at the reception, a preregistered picture (labels 610/620, see Lemmons, Figs. 6A and 6B) 
can be superimposed to a predetermined area of a moving object (soda can 608/618, see 
Lemmons, Figs. 6A and 6B), the system comprising: 

an estimator of the location, orientation and size of said predetermined area of said 
moving object in each video image (see Lemmons, [0081]-[0084], line 5), and 

a generator of a video stream in which each video image containing said area is attached 
to the selected information along with the location and size of the information area (see 
Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and page 12, lines 1-3). 
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Lemmons does not specifically disclose a memory storing a set of computer generated 
views of said picture for various orientations and associating with each computer generated view 
an orientation index that identifies the physical orientation of the computer generated view of the 
preregistered picture associated with the corresponding orientation index, 

wherein said set of computer generated views is received before the video images, 

a selector for selecting, among said set of computer generated views, a computer 
generated picture having the same orientation as said predetermined area in the video image as 
determined from orientation and position data captured by a sensor attached to the moving object 
and camera position data, and 

a generator of a video stream in which each video image containing said area is attached 
to the selected orientation index along with the location and size of the information area. 

In an analogous art, Bulman discloses a memory storing a set of computer generated 
views of said picture for various orientations (see Bulman, col 12, lines 21-25 and col 13, lines 
29-33) and associating with each computer generated view an orientation index that identifies the 
physical orientation of the computer generated view of the preregistered picture associated with 
the corresponding orientation index (see Bulman, Fig. 11 and col 12, lines 21-25 and col 13, 
lines 29-33), 

a selector for selecting, among said set of computer generated views, an computer 
generated picture having the same orientation as said predetermined area in the video image (see 
Bulman, col 13, lines 33-37 and Fig. 11.), and 

a generator of a video stream in which each video image containing said area is attached 
to the selected orientation index along with the location and size of the information area (The 
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orientation index of Bulman's system includes orientation and positioning information along 
with the different orientated views of the picture, see Bulman, col 13, lines 33-37. This then, 
reasonably corresponds to the data file 722 of Lemmons' system, which is transmitted with each 
image. Therefore, Lemmons in view of Bulman reasonably teaches the limitation of transmitting 
with each current image the selected orientation index along with information on the location and 
size of said predetermined area of said moving object (see Lemmons, [0084], lines 5-9 and '745, 
page 9, lines 14-29 and page 12, lines 1-3, and Bulman, col 13, lines 29-33). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify Lemmons' system to include the limitations as taught by Bulman for the 
advantage of providing an improved system for allowing a combined image to be displayed with 
a more natural look. 

Lemmons in view of Bulman does not specifically disclose receiving the views before the 
video images, 

orientation and position data captured by a sensor attached to the moving object, 
camera position data, 

wherein orientation is determined using the orientation and position data. 

In an analogous art, Wang discloses orientation and position data captured by a sensor 
(see Wang, col 7, lines 26-59) attached to the moving object (see Wang, col 7, lines 55-59), 

camera position data (see Wang, col 7, lines 8-12 and lines 60-66), and 

wherein orientation is determined using the orientation and position data (see Wang, col 
8, lines 12-16). 
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It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman to include the limitations as 
taught by Wang for the advantage of providing an improved system for allowing a combined 
image to be displayed with a more natural look, and further allowing the possibility to 
successfully integrate properly placed, scaled and computer generated synthetic content with 
video content. 

Lemmons in view of Bulman and Wang does not specifically disclose receiving the 
views before the video images. 

In an analogous art, Lemmons '981 discloses transmitting enhancement information in 
advance of transmission of a stream of video images (see [0029]-[0030], lines 1-4). The system 
of Lemmons in view of Bulman teaches providing enhancements to video content (see 
Lemmons, [0003], and Bulman, col 6, lines 6-10) in a more normalized (see Bulman, col 1, lines 
60-63), flexible and inexpensive manner (see Lemmons, [0006] and [0007]) in that the 
enhancement information does not have to be embedded within the video stream (see Lemmons, 
[0010]). Lemmons '981 teaches enhancement data being transmitted and received before the 
video stream to be enhanced is transmitted and received (see Lemmons '981, [0029]-[0030], 
lines 1-4). Lemmons '981 teaches enhancement data including stock updates and news stories, as 
well as other information, as examples only. It is well known in the art that enhancement data 
may be of any type or form, as shown in Lemmons and Bulman. 

Therefore, it would have been obvious for a person having ordinary skill in the art at the 
time of the invention to modify the system of Lemmons in view of Bulman to include the 
limitations as taught by Lemmons '981 for the advantage of providing an improved system for 
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delivering enhancements such that the enhanced video is presented in a more normalized, 
flexible and inexpensive manner, while conserving bandwidth. 

Regarding claim 13, Lemmons in view of Bulman, Wang and Lemmons '981 discloses a 
video receiver adapted to receive images (see Bulman, col 12, lines 21-25 and col 13, lines 29- 
33) comprising: 

the memory (see Bulman, col 12, lines 21-25 and col 13, lines 29-33); 

an extractor for from said memory an oriented picture on the basis of the orientation 
index attached to each video image of the video stream (see Bulman, Fig. 1 1 and col 12, lines 
21-25 and col 13, lines 29-37); and 

a calculator for providing a scaled picture on the basis of the size information attached to 
each video image in the video stream (see Lemmons, [0082]-[0084], line 5, Figs. 6A-6B and 
'745, Fig. 7 and page 5, lines 1-9), and for superimposing said scaled picture in the video image 
at the location corresponding to said location information (see Lemmons, Figs. 6A-6B and 
[0082]-[0083], line 5, and '745, Fig. 7 and page 5, lines 1-9). 

Regarding claim 14, Lemmons in view of Bulman, Wang and Lemmons '981 discloses a 
second set of views contains picture frames of same orientation of the first set of views, with a 
picture content (the second set of views are the updates to the hot spots, see Lemmons, [0010], 
lines 10-18, [0013], lines 6-14 and [0040], lines 6-7 and Figs. 6A-6B). 

Regarding claim 15, Lemmons in view of Bulman, Wang and Lemmons '981 discloses in 
which the first set of computer generated views contains only picture frames (because Lemmons 
teaches that an object may be defined with a hot spot of which may only include an outline of a 
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specific area, it is fairly suggested that a first set of views contains only picture frames, as there 
is no specific ad or label visible, see '745, page 5, lines 4-24). 

Regarding claims 16 and 18, Lemmons in view of Bulman, Wang and Lemmons '981 
discloses superimposing, with a video receiver, the computer generated view having the same 
orientation as said predetermined area in the current image in the video image (see Lemmons, 
Figs. 6A-6B and [0082]-[0083], line 5, and '745, Fig. 7 and page 5, lines 1-9 and Bulman, col 
13, lines 33-37). 

Regarding claim 20, Lemmons discloses a method for transmitting a stream of video 
(scenes/frames of the video, see Lemmons, [0031], lines 1-2 and Figs. 6A and 6B) such that a 
preregistered picture (labels 610/620, see Lemmons, Figs. 6A and 6B) can be superimposed to a 
predetermined area of a moving object depicted in the stream (soda can 608/618, see Lemmons, 
Figs. 6A and 6B), the method comprising: 

for each video image of the stream of video images: 

determining location, orientation and size of the predetermined area of the moving object 
in the video image (see Lemmons, [0083] -[0084], line 5); 

transmitting the video image along with the determined location and the size of the 
predetermined area of the moving object in the video image (see Lemmons, [0084], lines 5-9 and 
'745, page 9, lines 14-29 and page 12, lines 1-3). 

Lemmons does not specifically disclose receiving a first set of computer generated views 
of the preregistered picture in various orientations; 

in advance of transmission of the stream of video images, transmitting each computer 
generated view of the first set of computer generated views in association with an orientation 
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index that identifies a physical orientation of the computer generated view of the preregistered 
picture; 

determining orientation an position data captured by a sensor attached to the moving 
object and camera position data; 

selecting, from the orientation indices associated with the first set of computer generated 

views, 

an orientation index corresponding to an orientation of the predetermined area of the 
moving object in the video image, or 

transmitting with each current image the selected orientation index. 

In an analogous art, Bulman discloses receiving a first set of computer generated views of 
the preregistered picture in various orientations (see Bulman, col 12, lines 21-25 and col 13, lines 
29-33); 

selecting, from the orientation indices associated with the stored computer generated 
views, an orientation index of the computer generated view having the same orientation as a 
predetermined area of an object in a current image (see Bulman, col 13, lines 33-37 and Fig. 11.), 
and 

transmitting with each current image the selected orientation index (The orientation index 
of Bulman' s system includes orientation and positioning information along with the different 
orientated views of the picture, see Bulman, col 13, lines 33-37. This then, reasonably 
corresponds to the data file 722 of Lemmons' system, which is transmitted with each image. 
Therefore, Lemmons in view of Bulman reasonably teaches the limitation of transmitting with 
each current image the selected orientation index along with information on the location and size 
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of said predetermined area of said moving object (see Lemmons, [0084], lines 5-9 and '745, page 
9, lines 14-29 and page 12, lines 1-3, and Bulman, col 13, lines 29-33). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify Lemmons' system to include the limitations as taught by Bulman for the 
advantage of providing an improved system for allowing a combined image to be displayed with 
a more natural look. 

Lemmons in view of Bulman does not specifically disclose determining orientation and 
position data captured by a sensor attached to the moving object and camera position data, or 

transmitting the enhancement information in advance of transmission of the stream of 
video images. 

In an analogous art, Wang discloses determining orientation and position data captured 
by a sensor (see Wang, col 7, lines 26-59) attached to the moving object (see Wang, col 7, lines 
55-59) and camera position data (see Wang, col 7, lines 8-12 and lines 60-66), and 

wherein orientation is determined using the orientation and position data (see Wang, col 
8, lines 12-16). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman to include the limitations as 
taught by Wang for the advantage of providing an improved system for allowing a combined 
image to be displayed with a more natural look, and further allowing the possibility to 
successfully integrate properly placed, scaled and computer generated synthetic content with 
video content. 
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Lemmons in view of Bulman and Wang does not specifically disclose transmitting the 
enhancement information in advance of transmission of the stream of video images. 

In an analogous art relating to a system for presenting an enhanced image, Lemmons 
'981 discloses transmitting enhancement information in advance of transmission of a stream of 
video images (see [0029]-[0030], lines 1-4). The system of Lemmons in view of Bulman teaches 
providing enhancements to video content (see Lemmons, [0003], and Bulman, col 6, lines 6-10) 
in a more normalized (see Bulman, col 1, lines 60-63), flexible and inexpensive manner (see 
Lemmons, [0006] and [0007]) in that the enhancement information does not have to be 
embedded within the video stream (see Lemmons, [0010]). Lemmons '981 teaches enhancement 
data being transmitted and received before the video stream to be enhanced is transmitted and 
received (see Lemmons '981, [0029]-[0030], lines 1-4). Lemmons '981 teaches enhancement 
data including stock updates and news stories, as well as other information, as examples only. It 
is well known in the art that enhancement data may be of any type or form, as shown in 
Lemmons and Bulman. 

Therefore, it would have been obvious for a person having ordinary skill in the art at the 
time of the invention to modify the system of Lemmons in view of Bulman to include the 
limitations as taught by Lemmons '981 for the advantage of providing an improved system for 
delivering enhancements such that the enhanced video is presented in a more normalized, 
flexible and inexpensive manner, while conserving bandwidth. 

Regarding claim 21, Lemmons in view of Bulman and Lemmons '981 does not 
specifically disclose transmitting a polygon representation of an obstruction with the video image 
and the selected orientation index. 
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In an analogous art, Wang discloses transmitting a polygon representation of an 
obstruction with the video image and the selected orientation index (Wang teaches creating a 
view of a synthetic camera based on the field of view of an actual camera [see Wang, col 7, lines 
37-39]. Any object in a camera's field of view, then, including an obstruction, will be represented 
in the transmitted signal [see Wang, col 7, lines 37-54]. Furthermore, objects presented in the 
synthetic view may be three dimensional graphic representations of an object [see Wang, col 8, 
lines 43-52], which, then, must include polygons since virtual and/or 3D graphics may be built 
based on various polygons. Therefore, the limitation as claimed is met). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman and Lemmons '981 to include 
the limitations as taught by Wang for the advantage of providing an improved system for 
allowing a combined image to be displayed with a more natural look, and further allowing the 
possibility to successfully integrate properly placed, scaled and computer generated synthetic 
content with video content. 

8. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lemmons (of 
record) in view of Bulman (of record), Wang (of record) and Lemmons '981 (of record), as 
applied to claim 1 above, and further in view of Wixson (of record). 

Regarding claim 10, Lemmons in view of Bulman, Wang and Lemmons '981 does not 
specifically disclose static points of an image are localizable to detect when a new object comes 
into a next image. 
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In an analogous art relating to a system for detecting and tracking objects, Wixson 
discloses static points of an image are localizable to detect when a new object comes into a next 
image (see Wixson, col 1, lines 26-32). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman, Wang and Lemmons '981 to 
include the limitations as taught by Wixson, for the advantage of providing an advertisement to 
the newly detected object. 

9. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lemmons (of 
record) in view of Bulman (of record), Wang (of record) and Lemmons '981 (of record), as 
applied to claim 1 above, and further in view of Martinolich (of record). 

Regarding claim 17, Lemmons in view of Bulman, Wang and Lemmons '981 does not 
specifically disclose superimposing, with a video production mixer, the computer generated view 
having the same orientation as the area in the current image in the current image. 

In an analogous art relating to a system for the production of interactive video, 
Martinolich discloses superimposing, with a video production mixer, the computer generated 
view having the same orientation as the area in the current image in the current image (see 
Martinolich , [0020], lines 1-8, [0022], lines 1-19 and Fig. 1). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman, Wang and Lemmons '981 to 
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include the limitations as disclosed by Martinolich, for the advantage of using a commonly know 
and widely used device that allows secondary information to be added to a video signal. 

Conclusion 

10. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHENEA P. SMITH whose telephone number is (571)272-9524. 
The examiner can normally be reached on 8:00 am - 4:00 pm, Mon-Fri. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, William Trost can be reached on 571-272-7872. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Seivice Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/William Trost/ 

Supervisory Patent Examiner, Art Unit 
2421 

/CHENEA P SMITH/ 
Examiner, Art Unit 2421 



